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ABSTRACT 

The  ACT  Operational  Analysis  Workshop  has  been  held  yearly  since  2007  and  fulfils  in  a  need  to  bring 
together  the  NATO  Operational  Analysis  community  to  discuss  the  common  threads  on  the  application  of 
scientific  methods  to  support  the  decision  making  process  in  the  various  NATO  commands  and  agencies  and 
at  the  national  level.  While  Operational  Analysis  (OA)  is  applied  at  various  decision  making  levels,  the 
commonality  of  the  practice  of  OA  makes  it  possible  but  also  necessary  to  discuss  challenges,  exchange  best 
practices,  gain  insights  and  think  about  potential  improvements.  One  of  its  main  objectives  is  to  compare  the 
use  of  OA  within  the  NATO  Allied  Command  Operations  (ACO),  the  NATO  Allied  Command  Transformation 
(ACT),  and  the  national  laboratories  and  technical  centres,  and  to  determine  how  to  improve  synergy, 
cohesion  and  coordination  between  Operational  Analysts  within  the  NATO  command  and  force  structure  and 
the  nations.  Furthermore,  the  OA  Workshop  explores  opportunities  to  cooperate  between  NATO  and  national 
entities.  The  themes  of  the  three  workshops  so  far  have  been  very  diverse,  but  all  have  been  within  the  overall 
aim  “To  coordinate  and  improve  the  contributions  of  Operational  Analysis  to  NATO  operations  and 
capability  development 

This  paper  will  discuss  the  contribution  of  the  three  workshops  to  the  continuing  challenges  that  the 
operational  environment  and  the  transformational  agenda  is  being  confronted  with  and  touches  upon  the 
expectations  of  what  the  fourth  workshop  to  be  held  in  June  2010  will  bring.  Some  of  the  greatest  challenges 
include  dealing  with  the  complexity  of  the  current  operational  environment  with  less  emphasis  on  kinetic 
solutions.  Non-military  dimensions  and  out-of-area  operations  have  made  clear  that  more  analytical 
resources  need  to  be  applied  towards  “softer”  analytical  methods  making  less  use  of  mathematical 
optimization,  statistical  application  and  “number  crunching  Equally  challenging  is  the  problem  of  using 
experimentation  to  better  understand  and  evaluate  military  capabilities,  and  how  to  support  concept 
development  with  analytical  efforts.  The  workshops  have  been  striving  to  find  ways  to  improve  the  value  that 
OA  brings  to  decision-makers  in  operations  and  in  capabilities  development. 
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1  INTRODUCTION 

From  the  successes  of  applying  scientific  methods  to  decision  making  and  military  problems  during  World 
War  II,  the  field  of  Operational  Analysis  (OA)  rapidly  grew  into  a  successful  profession  that  supported 
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decision  making  in  a  variety  of  industries,  organisations  and  governments.  Within  the  military,  OA  remained  a 
speciality  discipline  and  many  defence  organisations  in  nations  have  created  OA  capabilities  to  support 
equipment  procurement,  capability  development  and  military  operations. 

The  OA  community  in  NATO  is  diverse,  with  staff  employed  to:  provide  assessment  of  progress  of 
operations;  support  concept  development  and  experimentation;  analyse  lessons  learned,  and  optimise  and 
evaluate  training,  to  name  a  few.  Given  this  diversity  in  application  of  structured  and  scientific  decision 
making  support,  many  staff  felt  that  a  regular  meeting  was  required  to  establish  a  sense  of  community  and 
professional  identity. 

Following  the  expression  of  this  requirement,  on  13  -  16  March,  2007,  at  the  Little  Creek  Conference  Centre, 
Norfolk,  the  OA  Branch  at  Headquarters  Supreme  Allied  Commander  Transformation  (HQ  SACT)  hosted  the 
first  NATO-wide  OA  Workshop  that  brought  together  many  analysts  from  throughout  the  NATO  structure. 
The  focus  was  on  establishing  an  analysis  community  and  attendees  were  called  to  give  briefs  on  their  current 
work  and  responsibilities.  A  good  understanding  of  the  wide  variety  of  NATO  analysis  ongoing  and  current 
practices  was  received,  and  a  realisation  that  many  of  those  current  practices  needed  improvement. 

On  17  November,  2008,  the  HQ  SACT  OA  Branch  hosted  the  second  NATO  OA  Workshop,  as  a  pre-event  to 
the  Concept  Development  and  Experimentation  conference  in  Oslo,  Norway.  This  time,  the  focus  was 
primarily  on  campaign  assessment.  It  was  realised  that  there  was  a  serious  problem  with  the  way  in  which 
NATO  assesses  progress  as  a  result  of  the  lack  of  both  standardised  processes  and  connections  between  staffs 
at  strategic,  operational  and  more  tactical/in-theatre  levels. 

NATO  HQ  SACT  hosted  the  3ld  Annual  NATO  Operational  Analysis  Workshop  on  9  -  1 1  June,  2009,  at  the 
Norfolk  Naval  Base.  The  core  theme  of  this  event  was  utilisation  of  analysis:  how  to  ensure  that  analytical 
products  from  defence  experimentation,  analysis  activities  supporting  operations,  and  lessons  learned  are  fully 
used  by  customers  and  stakeholders  in  these  respective  communities.  A  critical  question  that  is  intimately  tied 
with  the  issue  of  utilisation  was  also  raised:  the  existence,  definition  and  membership  of  the  analysis 
community. 

In  2010,  HQ  SACT  will  host  the  4th  OA  Workshop  from  15-17  June  2010.  The  theme  of  this  event  is  to 
examine  the  problems  that  Concept  Developers  face  and  the  role  of  analysis  in  supporting  Concept 
Development.  Within  NATO,  Concepts  are  a  primary  mechanism  for  initiating  and  debating  change,  yet  in 
comparison  to  Experiments,  it  has  been  challenging  to  apply  a  wide  variety  of  analytical  techniques.  It  is  our 
intention  to  determine  analysis  requirements  and  fit  analytical  methods  to  problems.  The  outputs  will  be  used 
to  inform  best  practice  for  Concepts  Development  within  NATO. 

The  purpose  of  this  paper  is  to  give  an  overview  of  the  aim  of  the  OA  workshops  and  to  discuss  how  an  event 
as  the  OA  workshop  can  influence  the  discussion  of  problems  within  NATO,  how  the  OA  workshop  functions 
as  a  gathering  of  likeminded  people  and  what  the  fall-out  of  the  meetings  are  for  the  OA  community  and  for 
NATO  as  a  whole. 

2  CONDUCT  OF  THE  WORKSHOPS 
2.1  Aim 

All  four  workshops  have  been  constructed  with  a  similar  aim  and  set  of  objectives  that  have  remained 
reasonably  robust  over  the  years.  While  there  are  some  differences  in  the  exact  formulation  of  the  aim  of  the 
workshops,  the  general  genesis  of  the  aim  has  been: 
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•  To  coordinate  and  improve  the  contributions  of  Operational  Analysis  to  NATO  operations  and 
capability  development. 

Within  that  context  the  contributions  from  the  OA  practitioners  within  the  Allied  Command  Operations 
(ACO)  concentrated  on  how  OA  supported  the  NATO  operations  in  Afghanistan,  Balkans,  off-the -coast  of 
Sudan,  the  Pakistan  earthquake  relief  efforts,  and  Operation  Active  Endeavour.  The  OA  contributions  in 
capability  development  have  been  concentrated  in  Allied  Command  Transformation  mainly  to  support 
Defence  Planning,  Concept  Development  and  Experimentation,  and  Research  and  Technology.  Analytical 
efforts  in  NATO  Agencies  have  been  in  support  of  ACO  and  ACT  in  these  areas. 

2.2  Objectives 

The  Operational  Analysis  efforts  faced  various  challenges  in  conducting  this  support,  and  the  OA  Workshop’s 
objectives  were  to  understand  these  challenges  from  the  various  entities  and  see  how  coordination  between  the 
Operational  Analysts  could  assist  in  finding  solutions  for  them.  The  objectives  for  the  workshops  therefore 
have  also  been  very  constant  from  2007  onwards: 

•  Review  the  various  ways  that  Operational  Analysis  is  currently  contributing  to  NATO  (commands 
and  nations)  operations  and  capability  (looking  at  ACO,  ACT  and  the  nations); 

•  Discuss  challenges,  best  practices,  external  insights  and  potential  improvements; 

•  Determine  how  to  improve  synergy,  cohesion  and  coordination  between  the  Operational  Analysts 
within  NATO  command  and  force  structure  and  the  nations; 

•  Identify  additional  challenges  and  opportunities  for  the  OA  Community  to  address  together  for  the 
good  of  the  Alliance. 

However,  the  actual  conduct  of  the  workshops  has  been  quite  different.  While  the  2007  workshop  served  as 
an  introduction  to  all  NATO  command  and  agencies  of  what  kind  of  activities  the  Operational  Analysts  were 
involved  in  and  sought  to  define  what  kind  of  mechanisms  could  improve  the  coordination  between  the 
various  entities,  the  2008  workshop  focused  on  the  single  topic  of  Assessment.  In  2009,  the  emphasis  was  on 
how  analysis  supported  three  different  strands  of  work  in  NATO,  and  in  2010,  the  focus  will  be  on  how 
operational  analysis  interacts  with  the  single  activity  of  Concept  Development  and  attempts  to  build 
understanding  with  non-analysts  on  what  OA  brings.  The  following  sections  provide  details  on  the  topics  and 
the  outcome  of  each  of  the  workshops. 


3  OA  WORKSHOP  2007 

The  2007  OA  workshop  was  an  opportunity  for  all  NATO  entities  to  present  on  current  work  and  methods,  for 
individual  OA  analysts  to  present  a  topic  or  study  result,  and  for  the  forming  of  syndicates  to  discuss  the 
future  of  OA.  At  that  time,  Operational  Analysts  were  situated  in  all  the  ACO  Joint  Force  Commands  (JFC), 
their  Air,  Land  and  Maritime  Component  Commands  (CC),  at  HQ  SACT,  at  Joint  Warfighting  Centre  (JWC), 
at  Joint  Analysis  Lessons  Learned  Centre  (JALLC),  at  the  NATO  Consultation  Command  &  Control  Agency 
(NC3A)  and  at  the  NATO  Undersea  Research  Centre  (NURC).  Additionally,  in  the  NATO  Force  Structure, 
the  Corps  HQs  had  OA  experts. 
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3.1  Current  Work 

A  full  day  of  the  workshop  was  devoted  to  these  HQs  and  Agencies  to  brief  on  their  OA  experiences,  what 
they  were  working  on,  and  how  their  advice  and  expertise  was  used  by  their  superiors.  The  focus  of  the  ACO 
JFCs  and  CCs  naturally  was  on  supporting  the  operational  planning  for  the  NATO  Operations  in  Afghanistan 
(ISAF),  Kosovo  (KFOR),  Pakistan  (Earthquake  Relief)  and  in  the  Mediterranean  (Operation  Active 
Endeavour),  and  on  assistance  to  the  NATO  Training  Mission  in  Iraq,  Hurricane  Katrina  relief  and  monitoring 
Dafur.  Emphasis  was  placed  on  supporting  the  then  developing  Effects  Based  Thinking,  and  on  campaign  and 
operational  assessment  through  the  measuring  of  effectiveness  and  progress  of  missions.  Their  work  was 
further  broadened  to  support  the  same  functions  in  the  NATO  Response  Force  exercises  where  substantial 
efforts  were  spent  in  supporting  the  course  of  actions  by  wargaming  with  NC3A  developed  tools.  On  the 
ACT  side,  analysts  in  various  HQ  SACT  branches  supported  the  development  of  requirements  -  hand  in  hand 
with  NC3A  -  for  the  Defence  Requirements  Review  (DRR)  and  Long  Term  Capability  Requirements 
(LTCR),  provided  strategic  analysis  for  ACT’s  and  Bi-SC  Seminars  and  Conferences,  conducted  the  analysis 
for  Concept  Development  and  in  Experiments,  while  the  JWC  analysis  concentrated  on  CD&E  projects,  the 
JALLC  produced  various  reports  on  burning  issues  in  NATO’s  operations  and  exercises,  and  NURC  reported 
on  OA  support  to  projects  in  port  protection,  autonomous  underwater  vehicles  and  maritime  situational 
awareness. 

3.2  Future  of  OA  in  NATO 

Keynote  addresses  from  Dr.  George  Rose  (UK)  and  Mr.  Michael  Bauman  (US)  provided  a  welcome 
introduction  into  how  two  major  NATO  nations  undertook  transformation  of  their  analysis  focus  to  the  future. 
Dr.  Rose  spoke  about  the  need  to  go  back  to  the  roots  of  OA  by  collecting  data  and  forward  deploy  analysts 
together  with  the  troops  to  learn  of  the  operational  problems  and  to  support  them  with  science  and  technology 
(S&T),  either  directly,  or  by  reach-back  to  the  UK  Defence  Science  and  Technology  Lab  (DSTL).  Mr. 
Bauman  spoke  about  the  need  for  OA  to  team  up  with  the  Subject  Matter  Experts  and  to  engage  major 
stakeholders,  while  maintaining  OA  integrity  and  professionalism,  to  support  the  development  of  concepts. 

Four  syndicates  were  formed  to  discuss  the  future  of  Operational  Analysis  in  NATO.  It  has  been  perceived 
that  the  role  of  the  operational  analyst  is  to  work  in  support  of  the  projects  and  staff  work  in  the  HQs,  and 
therefore  is  mainly  invisible  or  in  the  background.  The  lack  of  procedures,  vision,  or  a  recognized  program  of 
work  may  undercut  the  ability  of  maintaining  analytical  expertise  specifically  in  times  when  the  old  paradigms 
of  cold-war  analysis  have  become  obsolete  and  new  methods  and  tools  have  to  be  developed  to  be  able  to 
support  the  current  type  of  missions  of  NATO.  A  need  to  explain  the  analytical  method,  to  gather 
requirements  for  analytical  work,  to  learn  new  methodology  and  to  share  locally  developed  models,  was 
discussed  in  each  of  the  four  syndicates: 

•  NATO  Operational  Analysis  Handbook 

•  Analysis  Requirements  List 

•  Operational  Analysis  Education  and  Training 

•  Sharing  Analytical  Models 

The  four  syndicates  briefed  back  on  the  last  day  of  the  workshop  on  their  contribution  to  the  formulation  of  a 
common  analytical  approach  on: 

•  Principles  on  division  of  responsibilities  and  rules  of  cooperation  in  analytical  work  in  NATO; 
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•  An  agreement  to  consolidate  the  combined  NATO  requirements  for  Operational  Analysis  into  a  single 
Analysis  Requirements  List  (ARL)  and  initiate  the  formulation  of  business  rules  for  the  use  and 
implementation  of  the  ARL; 

•  Draft  an  outline  of  a  NATO  Operational  Analysis  Handbook  and  an  agreement  on  sharing  methods, 
models,  database  formats,  dictionaries  and  contents; 

•  Principles  on  conducting  peer  review  of  OA  work,  on  creating  training  opportunities  and  on  building 
a  NATO  OA/OR  network. 

3.3  Special  Topics 

During  the  2007  workshop  there  was  also  the  opportunity  to  present  analytical  work  done  on  special  topics. 
These  special  topics  were  grouped  under  four  headings: 

•  OA  in  Concept  Development  and  Experimentation 

•  OA  to  support  Operational  Plans 

•  OA  Case  Studies 

•  OA  in  Campaign  Assessment 

Two  presentations  on  CD&E  discussed  how  analysis  supports  Concept  Development  through  problem 
structuring,  formulation,  brainstorming,  simple  modelling  and  risk  and  feasibility  analysis,  and  how  vital  it  is 
to  insert  the  analyst  into  the  design  of  the  experiment  from  the  beginning  to  ensure  that  the  aims  and 
objectives  of  the  experiment  can  be  met  by  a  well-designed  data  capturing  and  analysis  plan.  Examples  were 
given  for  the  concepts  of  Joint  Urban  Operations  (JUO)  and  Joint  Intelligence  Surveillance  and 
Reconnaissance  (JISR)  and  the  Multinational  Experiments  (MNE)  on  the  Effects  Based  Approach  to 
Operations  (EBAO).  A  third  presentation  detailed  the  needs  for  proper  survey  and  questionnaire  design. 

The  second  group  contained  a  briefing  on  the  NC3A  model  GAMMA  (Global  Aggregated  Model  for  Military 
Assessment)  and  how  it  was  used  in  a  reach-back  mode  to  support  the  JFCs  in  planning.  NC3A  further 
briefed  on  the  models  that  are  used  in  the  DRR  project  to  determine  the  Minimum  Military  Requirement  for 
forces  over  the  medium  term  informing  the  Defence  Planning  Process  on  force  structures  required  from  the 
nations.  The  Air  CC  Izmir  briefing  was  on  a  network  optimization  model  used  to  plan  for  humanitarian  relief 
in  a  NATO  consequence  management  mission. 

The  case  study  on  social  network  analysis  having  potential  uses  in  analyzing  Command  and  Control 
vulnerabilities,  situational  awareness,  lessons  learned,  concept  development  etc  highlighted  the  capabilities  of 
a  new  method  to  be  used  in  NATO  analysis.  A  presentation  on  logistics  regression  provided  a  way  to  cope 
with  binary  data  analysis  resulting  from  data  collection  through  surveys.  A  analysis  of  the  tasks  that  need  to 
be  conducted  in  countering  piracy  of  the  Horn  of  Africa  used  a  mind  mapping  method  to  brainstorm  and 
structure  the  problem. 

Campaign  assessment  in  exercises  and  operations  faces  a  number  of  challenges,  current  practices  and  methods 
to  tackle  the  problem  were  discussed  in  three  separate  briefs  on  the  topic. 

3.4  Conclusion 

The  2007  OA  workshop  concluded  with  promises  to  develop  an  OA  handbook,  to  build  a  NATO  OA 
community,  to  develop  a  consolidated  ARL  and  to  look  for  opportunities  to  develop  an  introductory  course  for 
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military  staffers  that  are  placed  in  an  OA  billet  or  aspire  to  conduct  analysis  to  appreciate  and  understand 
principles  of  analysis. 


4  OA  WORKSHOP  2008 

While  the  2007  OA  workshop  was  conducted  over  three  days,  the  choice  to  conduct  in  2008  a  one-day  OA 
workshop  on  the  role  of  analysis  in  supporting  Campaign  Assessment  at  both  the  Strategic  and  Operational 
levels,  was  given  by  the  urgent  need  by  the  ACO  commands  to  change  their  approach  to  assessment  in  light  of 
developments  in  ISAF.  The  one-day  workshop  was  an  add-on  to  the  CD&E  conference,  but  only  for 
organizational  purposes  and  to  maximize  participation.  The  workshop  allowed  for  participation  by  national 
representatives  from  NATO  nations,  who  were  either  invited  through  the  mechanisms  of  the  RTO  or  had 
professional  links  with  the  topic  assessment.  The  workshop  covered  in  two  sessions  the  role  of  analysis  in 
supporting  assessments  at  the  operational  level  and  at  the  strategic  level.  Each  session  was  introduced  by  two 
presentations  after  which  a  plenary  debate  under  the  direction  of  a  panel  discussed  the  following  topics: 

•  What  is  the  state-of-the-art  with  respect  to  tools  and  techniques  for  conducting  campaign 
assessments? 

•  What  are  the  challenges  facing  analysts  conducting  analysis  at  Operational  and  Strategic  levels? 

•  What  are  the  relationships  between  Operational  and  Strategic  level  assessments:  are  there  hierarchical 
links  or  are  they  independent? 

4.1  Urgent  Need 

The  urgent  need  to  discuss  assessment  was  given  by  operational  considerations  that  were  the  topic  of 
discussions  in  which  SACEUR  himself  was  involved.  Finally,  in  December  2007  SACEUR’s  Strategic 
Commanders’  Conference  identified  a  need  for  a  continuing  Strategic  Assessment  process  to  inform  SACEUR 
of  the  progress  of  the  ISAF  mission1.  As  a  result  an  Urgent  Operational  Requirement  for  an  ISAF  Strategic 
Assessment  Capability  (ISAC)  was  drafted  by  SHAPE  and  was  approved  by  the  Military  Committee  in 
August  20082.  The  final  solution  for  the  ISAC  includes  programmatic  support  for  the  Campaign  Assessment 
Tool  developed  by  NC3A  and  calls  for  a  dedicated  analyst  support  in  situ  in  ISAF. 

4.2  Presentations 

The  first  session  contained  a  Netherlands  presentation  on  how  Influence  Diagrams  were  used  to  assess  the 
operation  of  the  Dutch  ISAF  contingent  in  Uruzgan.  The  tool  used  provided  useful  information  and  in  concert 
with  the  data  collection  done  by  the  embedded  Operational  Analysts,  allowed  for  a  comprehensive  national 
assessment  of  the  operation  in  Uruzgan.  The  presentation  of  the  developments  on  the  NATO  Assessment 
Handbook  produced  by  a  sub  group  that  is  part  of  the  Bi-SC  EBAO  Working  Group,  gave  the  participants  an 
appreciation  of  work  progress  on  assessment  since  the  introduction  of  the  EBAO  concept.  The  handbook 
outlines  how  the  measures  of  effectiveness  and  measures  of  performance  are  constructed  and  derived  from  the 
formulation  of  effects  and  actions,  and  gives  guidance  on  how  to  construct  them.  The  second  session  on 
strategic  level  assessment  provided  a  brief  on  the  ISAF  Strategic  Assessment  Capability  itself,  and  showed 
that  the  method  used  was  very  much  tailored  to  the  ISAF  situation.  A  subsequent  presentation  warned  for  the 
oversimplification  of  aggregation  of  assessment  results,  the  implicit  cause  and  effect  relations  that  some  of  the 


1  ISAF  Strategic  Assessment  -  Briefing  to  the  Bi-SC  Staff  Talks,  December  2007 

MCM-0 104-2008  -  Minimum  Military  Requirement  for  ISAC  Initial  Operational  Capability,  27  August  2008 
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operational  plans  contain,  and  the  difficulty  to  design  and  construct  measures  of  effectiveness  and 
performance,  especially  when  the  operational  plan  already  had  been  constructed  without  consideration  of 
these  measures. 

4.3  Plenary  Discussion  and  Conclusion 

The  workshop  concluded  that  the  overall  the  implementation  of  assessment  is  inconsistent  across  NATO: 
where  there  are  dedicated  resources  and  expertise  then  there  are  areas  of  good  practice.  There  is  a  continuing 
need  for  Campaign  Assessment  within  NATO,  in  line  with  Comprehensive  Political  Guidance3,  but  an  overall 
concept  to  ensure  robust  delivery  of  NATO’s  campaign  assessment  requirements  is  lacking.  It  was 
recommended  to  develop  an  Assessment  Concept  that  should  be  based  on  an  integrated  process  that  delivers 
products  at  the  Tactical,  Operational,  Theatre  Strategic,  and  Grand  Strategic  levels,  focusing  on  Operational 
and  Theatre  Strategic  products  while  identifying  key  interfaces  between  all  products.  Products  would  be 
prepared  and  delivered  by  Assessment  staff,  and  supported  by  Analysis  staff  that  can  undertake  deeper 
analysis  where  necessary.  The  products  should  be  flexible  to  meet  the  appropriate  commander’s  needs,  but 
best  practice  should  be  identified  for  the  content  and  format.  The  Assessment  Concept  should  be  cognisant  of 
the  complexity  of  the  operating  environment.  This  has  implications  for  the  measures  selected,  the 
assumptions  pertaining  to  the  relationship  between  measures,  and  the  tools  and  techniques.  The  Assessment 
Concept  needs  to  be  coherent  across  DOTMLPFI  to  maximise  the  benefits  of  its  implementation.  The 
Assessment  Concept  should  be  a  Bi-SC  deliverable. 


5  OA  WORKSHOP  2009 

As  in  2008,  the  OA  workshop  extended  invitations  to  the  nations  through  the  R&T  Agency  (RTA)  Systems 
Analysis  and  Studies  (SAS)  Panel  representatives.  A  number  of  nations  attended  and  provided  briefings.  The 
2009  OA  workshop  was  constructed  around  three  broad  strands  of  OA  work,  each  having  their  own  sub¬ 
objectives: 

•  Analysis  in  Experimentation,  with  the  objective  to  explore  and  seek  improvements  to  the  OA 
contribution  to  operational  experimentation  in  NATO,  to  expose  the  entire  OA  community  to 
approaches  to  operational  experimentation  as  described  in  the  GUIDEx4,  the  Code  of  Best  Practices 
(COBP)  for  Experimentation5,  and  the  developing  NATO  CD&E  Guide,  and  to  discuss  among  a 
limited  group  of  experiment-focused  participants  the  successes,  challenges  and  potential 
improvements  to  OA’s  contribution  to  NATO  operational  experimentation; 

•  Systems  Analysis  in  Operations,  Planning  and  Assessment,  with  the  objective  to  explore  and  seek 
improvements  to  the  OA  contribution  to  systemic  analysis  in  support  of  planning  and  assessment  in 
NATO  operations  in  a  complex  operational  environment,  to  expose  the  entire  OA  community  to  the 
NATO  approaches  to  systems  analysis  (known  by  a  variety  of  systems  names)  as  it  is  applied  to 
supporting  operational  planners  and  decision-makers  in  NATO  operations,  and  to  discuss  among  a 
limited  group  of  systems-analysis  participants  the  successes,  challenges  and  potential  improvements 
to  OA’s  contribution  to  operational  planning,  assessing  and  decision-making; 


3 

Comprehensive  Political  Guidance,  7  December  2007,  Para  16b 

4  GUIDEx  -  Guide  for  Using  and  Implementing  Defense  Experimentation,  The  Technical  Cooperation  Program  (TTCP).  March 
2006 

5  David  Alberts  and  Richard  Hayes,  Command  and  Control  Research  Program  (CCRP)  -  Code  of  Best  Practices  for 
Experimentation,  Washington,  2002 
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•  Operational  Analysis  for  Capability  Improvement  (Lessons  Learned),  with  the  objective  to  explore 
and  seek  improvements  to  the  analytical  contributions  to  NATO’s  lessons  learned  processes,  primarily 
as  conducted  by  the  Joint  Centre  for  Operational  Analysis  (JCOA)  and  the  Joint  Analysis  and  Lessons 
Learned  Centre  (JALLC),  to  expose  the  entire  OA  community  to  the  NATO  approaches  to  analytical 
support  to  lessons  learned  and  the  way  results  are  incorporated  into  capability  improvements,  and  to 
discuss  among  a  limited  group  of  lessons  learned-focused  participants  the  successes,  challenges  and 
potential  ways  to  improve  analytical  contributions  to  operational  capabilities  and  capability 
development. 

5.1  Keynote  Addresses  and  Plenary  Presentations 

Major  General  Skare,  Assistant  Chief  of  Staff  for  Implementation  at  HQ  SACT  opened  the  meeting  in  a 
plenary  session,  reminding  the  audience  that  he  understands  “the  value  that  [the  analytical  community]  adds 
and  the  need  for  ensuring  that  [it]  is  relevant  today”.  However,  he  challenged  the  audience  to  define 
themselves  as  problem  solvers  not  as  people  that  have  a  tendency  to  “fall  in  love  with  the  problem.” 

Brigadier-General  Crutchfield,  Director  of  the  United  States  Joint  Centre  for  Operational  Analysis  (JCOA), 
provided  the  first  keynote  address  in  which  he  explained  how  his  organisation  has  been  more  successful  lately 
ensuring  the  utilisation  of  their  analytical  products.  Consequently,  the  JCOA  is  now  tasked  to  maximum 
capacity  to  analyse  a  number  of  critical  operational  problems  in  Iraq  and  Afghanistan.  He  pointed  out  that  a 
team  effort  between  the  staff  from  his  ‘Engagement’  division  and  the  ‘Studies  and  Analysis’  division,  has 
made  all  the  difference:  whilst  the  latter  produces  and  delivers  the  analysis  reports,  the  former  visits  all 
relevant  stakeholders  and  interested  customers  with  a  focused  brief  on  what  the  results  and  recommendations 
mean  for  their  particular  areas. 

The  second  keynote  address  was  given  by  Mr.  Paul  Labbe,  Head  of  Science  and  Technology  Capability 
Management  at  Defence  Research  and  Development  Canada,  and  chair  of  the  international  study  group  which 
drafted  the  GUIDEx.  The  GUIDEx  provides  the  foundation  for  experimentation  campaigns  at  ACT  and  in 
many  nations.  Mr.  Labbe  explained  that  experimentation  itself  must  be  seen  as  a  campaign  in  which  the 
problem  statement  is  investigated  through  concept  development,  analytical  games,  modeling  and  simulation, 
studies  and  analysis,  and  event-like  experiments,  in  which  equal  time  is  spent  between  understanding  the 
problem  and  solving  it.  He  showed  that  experimentation  is  a  team  effort  between  analysts,  scientists  and 
operators. 

The  morning  plenary  sessions  of  the  second  and  third  day  were  filled  with  a  challenging  and  diverse  set  of 
briefings:  the  role  of  operational  analysis  in  a  campaign  of  chemical,  biological,  radiological  and  nuclear 
detection  capability  concept  development  and  experimentation;  the  ability  of  and  modelling  and  simulation  to 
support  each  other;  the  nature  of  today’s  defence  problems  which  tend  to  be  “wicked”  and  complex  and  the 
challenges  this  brings;  a  novel  technique  of  disruptive  technology  games  to  develop  concepts;  the  NATO  ACT 
Multiple  Futures  project;  the  exploitation  of  work  from  the  NATO  Research  and  Technology  Organisation; 
and  the  new  NATO  Defence  Planning  Process. 

5.2  Analysis  in  Experimentation 

The  purpose  of  subdividing  the  workshop  into  three  strands  was  to  allow  analysts  to  present  their  work  to  a 
peer  group,  and  to  discuss  the  critical  issues  of  how  to  improve  utilisation  of  analysis  results  and  how  to 
maintain  an  analytical  community. 
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5.2.1  Analysis  Community  in  Experimentation 

Given  that  NATO  and  many  nations  are  involved  in  experimentation  and  commit  significant  resources,  money 
and  intellectual  development,  the  need  for  a  professional  forum  was  discussed.  Often  the  analytical  problems 
faced  by  analysts  and  experimenters  are  very  similar,  even  across  diverse  fields.  For  example,  the  issue  of 
how  to  measure  situational  awareness  has  arisen  in  many  past  experiments,  and  is  now  an  issue  for  upcoming 
experiments.  Performing  a  basic  literature  review  in  the  defence  experimentation  field  is  difficult,  as  a  good 
body  of  published  knowledge  is  not  available.  It  is  quite  typical  for  projects  to  start  from  scratch  and  reinvent 
the  wheel  with  regard  to  commonly  measured  defence  parameters  such  a  situational  awareness,  network 
robustness,  plan  effectiveness  and  information  timeliness — a  significant  waste  of  resources  and  time. 
Developing  a  strong,  professional  analysis  community  is  the  primary  way  in  which  to  resolve  this  situation. 
This  requires  several  activities  of  the  defence  experimentation  community: 

•  Knowledge  Sharing  in  Published  Literature:  Most  professional  fields  have  a  publication  in  which 
new  methods,  results  and  news  can  be  shared.  The  experimentation  community  should  be  encouraged 
to  publish  the  methods  and  results  of  concept  development  and  experimentation  campaigns  in  a 
professional  journal.  NATO  HQ  SACT  is  currently  investigating  the  possibility  of  creating  an  annual, 
peer-reviewed,  online  journal  with  the  NATO  Research  and  Technology  Organisation  or  the  US 
Command  and  Control  Research  Program. 

•  Regular  Conferences  and  Workshops:  Most  professions  have  regular  conferences  where  members 
can  network,  share  work,  educate,  discuss  common  problems,  and  most  importantly,  maintain  a 
professional  identity.  The  need  for  a  regular,  annual  conference  for  the  defence  experimentation 
community  was  reaffirmed. 

•  Training  and  Continuous  Education:  The  previous  two  points  will  address  the  need  for  continuous 
education  to  some  extent,  however,  there  is  little  in  the  way  of  suitable  training  for  the  field  of  concept 
development  and  experimentation.  For  example,  the  use  of  modelling  and  simulation  in  support  of 
concept  development  and  experimentation  is  not  well  established.  The  purposes  of  a  professional 
community  are  to  set  standards,  develop  best  techniques  and  promote  learning  -  the  overall  result  is 
better  quality  for  the  customer.  The  community  should  strive  to  seek  and  develop  appropriate  training 
materials  and  guides.  Although  the  HQ  SACT  teaches  an  introductory  course  at  the  NATO  School 
Oberammergau,  several  nations  are  interested  in  more  advanced  courses  focusing  on  methods. 

5.2.2  Experimentation  Issues 

The  issues  of  rigour  and  validity  in  experimentation  were  discussed.  A  set  of  scientific  principles, 
encapsulated  within  the  GUIDEx,  guide  experimenters  and  analysts  in  the  correct  and  scientific  method  of 
experimentation;  however,  these  are  difficult  to  maintain  with  the  many  organisational,  resource  and  time 
constraints  usually  present. 

Much  of  the  activity  in  experimentation  campaigns  does  not  fall  under  the  strict  scientific  definition  of 
‘experiment’,  however,  care  must  be  taken  to  distinguish  between  an  ‘experiment’  -  which  is  the  formal, 
hypothesis  test  to  establish  a  causal  relationship  by  showing  covariance  between  variables  in  a  controlled 
environment  -  and  ‘experimentation’  which  is  now  understood  in  a  broader  sense  as  the  wide  variety  of 
activities  that  contribute  to  the  overall  development  of  a  capability.  This  can  include  an  iterative  cycle  of 
paper  studies,  modelling  and  simulation,  observational  studies,  limited  discovery  experiments,  which  then 
may,  or  may  not  culminate  in  a  ‘full-blown’  scientific  experiment.  Although  organisational  limitations, 
budgets  and  politics  are  often  major  constraints,  the  idea  of  experimentation  campaigns  need  to  be 
emphasised. 
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5.2.3  Utilisation  of  Experimentation  Results 

Primarily,  the  way  in  which  results  from  experimentation  are  used  depends  on  a  complex  combination  of 
organisational  structural  issues,  leadership  and  management  and  budgetary  processes;  factors  that  are  difficult 
to  change  by  one  single  group  of  people.  However,  there  are  several  actions  that  analysts,  in  concert  with 
concept  developers  and  experimenters,  can  do  to  improve  utilisation. 

•  Focus  on  Conclusions  and  Recommendations:  Reports  should  be  ‘utilisation-focused’  from  their 
outset.  This  involves  carefully  structuring,  presenting  and  emphasising  the  recommendations,  and 
directing  them  towards  appropriate  customers  and  stakeholders. 

•  Focus  on  Gaining  Feadership  Support:  The  success  of  any  effort  ultimately  depends  on  critical 
support  of  key  leadership.  Flag  and  General  Officers  will  have  to  underscore  the  need  for 
experimentation  by  confirming  the  objectives,  re-assessing  the  concept  or  conceptual  framework,  and 
their  stake  in  the  experiment.  Without  this,  the  utilisation  of  results  will  fail. 

•  Focus  on  Marketing  Successes  and  Results:  Although  such  activities  are  not  typically  the  purview 
of  analysts  and  scientists,  the  power  of  good  marketing  must  be  accepted.  For  each  experiment, 
concept  development  or  analysis  study,  analysts  must  ensure  that  accompanying  ‘glossy’  brochures, 
websites,  weekly  updates,  posters,  and  tri-fold  leaflets  are  produced  in  order  to  spread  awareness  and 
gain  leadership  acceptance.  It  was  suggested  that  leadership  should  require  that  all  ongoing  and 
completed  work  is  published  in  command  quarterlies  or  magazines. 

5.3  Systems  Analysis  and  Assessment  in  Support  of  Operations 

As  more  nations  commit  significant  resources  in  Afghanistan  and  other  operational  theatres,  there  is  increased 
recognition  of  the  need  to  demonstrate  progress.  SACEUR  himself  testified  to  the  US  Senate  Armed  Services 
Committee  recently,  acknowledging  that  there  was  no  reliable  way  to  measure  progress  in  Afghanistan. 
Given  the  large  changes  implemented  at  the  operational  level  in  the  NATO  command  structure,  which 
involves  the  creation  of  ‘Assessment’  branches,  the  time  is  right  for  a  significant  improvement  in  the  way  in 
which  NATO  measures  operational  progress.  In  common  with  Assessment,  the  new  field  of  systems  analysis 
and  knowledge  development  is  requiring  changing  NATO  operational  structures  and  processes. 

5.3.1  Customer  Support  and  Understanding 

Given  that  both  Assessment  and  Systems  Analysis  are  relatively  new  subject  areas  in  NATO  and  many 
nations,  there  is  a  significant  need  for  basic  training  in  these  areas  for  customers  of  analysis  products  and  key 
leadership.  The  analytical  community  should  maintain  involvement  with  the  relevant  training  authorities  to 
ensure  the  quality  of  training  courses. 

5.3.2  Ongoing  Projects  in  ACT  and  ACO  to  Develop  the  NATO  Assessment  Process 

The  current  NATO  assessment  processes  are  is  in  the  process  of  being  developed.  The  2007  and  2008 
workshops,  in  which  almost  all  the  NATO  commands  were  represented,  confirmed  the  need  for  urgent  action 
on  creating  an  improved  assessment  process.  Currently,  the  processes  used  by  analysts  in  operations  are  ad- 
hoc  and  not  formalised  and  there  is  no  NATO  doctrine  on  which  to  base  practice.  The  publication  of  a  new 
NATO  operations  planning  process  should  go  some  way  towards  rectifying  this  situation,  although  much 
work  remains.  Supporting  software  tools  are  also  in  development  by  NC3A,  although  they  of  course  depend 
on  the  development  of  the  doctrine  and  processes.  The  Workshop  was  an  opportunity  to  demonstrate  the 
various  work  projects  ongoing  in  the  area  of  Assessment,  which  are  outlined  below. 
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•  Bi-Strategic  Command  Effects-Based  Approach  to  Operations  Working  Group  (Bi-SC  EBAO 
WG):  This  working  group  has  a  sub-working  group  which  is  refining  the  NATO  Assessment 
Handbook,  and  recently  began  work  on  the  Assessment  components  of  the  NATO  Operations 
Planning  Process  document. 

•  ISAF  Strategic  Assessment  Capability:  The  ISAF  Strategic  Assessment  Capability  (ISAC)  project 
was  created  at  SACEUR’s  request  to  rectify  the  deficiency  of  assessment  at  the  strategic  level. 
Although  there  are  many  beneficial  aspects  and  novel  aspects  to  this  project,  it  was  felt  that  this  work 
was  a  temporary  fix,  and  that  the  analysis  community  needs  to  work  towards  ensuring  that  the  project 
is  generalised  into  a  strategic  level  assessment. 

•  Multi-National  Experiment  6:  During  the  conference,  the  way  in  which  civilian  institutions  and 
organisations  such  as  the  United  Nations  perform  assessment  (“evaluation”)  was  discussed.  The  Bi- 
SC  EBAO  WG  is  considering  these  methods  in  the  development  of  the  NATO  Assessment  Handbook. 
A  brief  was  given  on  Multinational  Experiment  6  (MNE  6),  a  campaign  of  concept  development  and 
experimentation  which  is  examining  civilian  methods  and  looking  for  areas  where  the  military  can  use 
well  developed  techniques  and  practices.  All  Assessment  staffs  in  the  Allied  Command  Operations 
structure  are  encouraged  to  be  involved  in  this  program. 

•  NATO  Research  and  Technology  Organisation:  A  study  on  Assessment  (HFM-185)  has  been 
initiated  by  a  request  from  several  nations  (USA,  CAN,  GBR,  NLD,  DEU).  The  aim  of  the  study  is  to 
share  current  practices  and  best  practices  on  Assessment,  and  determine  where  these  can  be  applied  to 
NATO  practices. 

•  United  States  Measuring  Progress  in  Conflict  Environments  (MPICE):  Dr.  Barbara  Sotirin 
presented  the  work  of  her  team  at  the  US  Army  Corps  of  Engineers.  Although  a  national  project,  the 
work  is  well  developed  and  operationally  tested,  and  should  be  investigated  by  NATO  to  extract 
beneficial  and  applicable  aspects. 

5.3.3  Training 

Although  NATO  civilian  analyst  posts  are  filled  by  pre-qualified  experts,  there  is  no  such  guarantee  for  the 
many  military  posts  in  the  new  Assessment  and  Knowledge  Development  branches  in  the  Joint  Force 
Commands.  The  necessity  for  development  of  training  courses  in  Assessment  was  discussed,  as  there  is 
currently  no  formal  training  (outside  of  limited  instruction  for  exercises). 

5.4  Analysis  in  Support  of  Lessons  Learned  and  Capability  Improvement 

This  strand  featured  both  the  NATO  and  US  national  lessons  learned  process  and  a  wide  range  of  issues  were 
discussed. 

5.4.1  Use  of  Analysis  Products  in  Lessons  Learned 

Analysis  in  support  of  both  the  NATO  and  US  lessons  learned  process  generally  has  a  well-defined  customer 
from  which  initial  requirements  are  generated.  It  was  considered  that  analysis  products  are  used  effectively  to 
bring  about  change  across  the  DOTMLPFI6  lines  of  capability  development.  The  importance  of  keeping  the 
leadership  engaged  was  emphasised,  and  stakeholder  analysis  makes  an  important  contribution.  Although 
both  NATO  and  national  organisations  focus  on  operational  issues,  frequently  strategic  and  tactical  issues  are 


6  Doctrine,  Organization,  Training,  Materiel,  Leadership,  Personnel,  Facilities  and  Interoperability 
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addressed.  When  tactical  issues  are  observed  by  the  analysis  teams  in-theatre,  they  need  to  be  brought  to  the 
attention  of  the  local  Commander  as  soon  as  possible,  and  certainly  in  the  out-brief  as  the  team  departs  from 
theatre.  Discussions  on  training  needs  analysis  resulted  in  the  view  that  training  is  a  consumer  of  analysis 
products  that  demonstrate  the  gaps  in  training  (i.e.  the  T  in  DOTMLPF  recommendations).  The  JALLC 
Lessons  Learned  Database  was  viewed  as  a  useful  tool  to  be  able  to  communicate  information  and  analysis 
products  in  the  NATO  environment,  but  an  effective  search  engine  is  essential  for  information  management. 

5.4.2  Organisational  Complexity  and  Political  Considerations 

Organisational  complexity  adds  difficulty  to  the  handling  of  recommendations  made  in  analysis  reports  and 
often  results  in  time  delays.  It  was  noted  that  the  findings  and  recommendations  in  JALLC  analysis  reports, 
once  signed-off  by  the  JALLC  Commander,  are  sent  to  the  two  NATO  strategic  commands  for  endorsement. 
Although  not  ideal,  seeking  endorsement  by  HQ  SACT  and  then  by  SHAPE  results  in  additional  delays  in  the 
release  of  reports.  It  is  perceived  that  developing  relationships  between  the  different  entities  involved  in 
Lessons  Learned  implementation  processes  is  key:  as  stated  in  the  NATO  bi-strategic  command  lessons 
learned  directive,  the  key  principles  are:  cooperation,  coordination,  communication. 

Discussion  centred  on  the  fine  balance  between  making  findings  and  recommendations  “palatable”  and  being 
able  to  “tell  it  how  it  is”.  An  essential  skill  for  report  writers  is  being  able  to  write  in  a  politically  acceptable 
way,  yet  ensuring  that  the  context  is  not  lost. 

5.4.3  Stakeholder  Customer  Involvement  with  the  Analysis  Task 

As  generally  the  NATO  and  national  Lessons  Learned  processes  have  a  well-defined  customer,  this  was  not 
viewed  to  be  a  significant  issue;  however,  it  is  recognised  that  the  rotation  of  military  staff  may  change  the 
requirement  for  the  analysis  study.  Within  the  NATO  process  the  first  step  in  the  procedure  is  to  engage  with 
the  customer  to  clarify  the  analysis  requirement. 

The  key  issue  with  customer  involvement  is  seen  to  be  building  trust  between  the  customer,  identified 
stakeholders  and  the  analysis  team.  Stakeholder  analysis  at  project  initiation  is  seen  to  be  essential  part  of  this 
trust  building.  The  JALLC  uses  a  ‘Coordinating  Draff  report  distributed  to  all  identified  stakeholders  to  elicit 
customer  review  and  comment  before  publication  of  the  final  report.  The  analysis  team  must  not  be  viewed 
by  the  command  staff  as  being  evaluators:  the  aim  of  the  analysis  is  to  identify  those  factors  that  lead  to  lead 
to  systematic  errors,  not  to  evaluate  the  performance  of  the  staff. 

5.4.4  Presentation  of  Results 

The  presentation  of  analysis  results  in  a  way  that  assists  the  user  of  these  results  is  an  essential  aspect  to 
consider  when  considering  the  output  of  an  analysis  study  -  the  final  user  of  the  study  must  be  considered 
from  the  outset.  Writing  a  well  structured  report  with  an  executive  summary  is  fundamental.  For  example, 
JCOA  presents  results  in  a  matrix  of  recommendations  against  DOTMLPF  (+P)  lines  of  capability 
development,  and  JAFFC  adopts  the  same  approach  in  its  analysis  reports  where  appropriate.  It  was  noted 
that  the  US  system  can  accept  unformatted  entries  (e.g.  reports  can  simply  be  embedded)  whereas  the  current 
NATO  Lessons  Learned  database  is  constrained  to  the  “observation,  discussion,  conclusion  and 
recommendation  format”.  It  was  felt  that  a  “finding,  discussion  and  recommendation”  format  may  be  a  more 
natural  way  of  presenting  Lessons  Learned  analysis  results  and  it  was  agreed  that  this  is  an  approach  that 
should  be  investigated  by  HQ  SACT  and  the  JALLC  when  reviewing  the  user  requirements  for  the  new 
NATO  Lessons  Learned  database  software. 
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The  participants  noted  the  function  of  the  JCOA  Engagement  Division  in  providing  contact  and  the  push  of 
analysis  results  to  customers  and  stakeholders.  It  was  recognised  that  this  not  being  well  done  in  the  NATO 
Lessons  Learned  process  and  could  be  improved.  JCOA  analysis  reports  are  finally  signed  off  by  the  report’s 
customer  whereas  JALLC  reports  are  signed  off  by  the  Commander  JALLC  and  then  are  sent  to  the  Bi-SCs 
for  endorsement. 

5.5  The  NATO  Analysis  Community 

The  proposed  changes  in  NATO  structure  have  brought  a  difficult  challenge  to  the  Operational  Analysis 
community,  which  traditionally  (and  informally)  has  been  defined  in  NATO  as  those  expert  civilian  staff  who 
employ  structured  and  scientific  methods  to  support  decision-making  in  operations  in  all  fields.  Now, 
operational  analysis  staff  are  being  distributed  in  various  places  in  ACO  including  within  new  Assessment 
branches  and  in  new  Knowledge  Development  branches.  Additionally,  within  ACT,  Operational  Analysis 
manpower  has  been  cut.  Some  questions  now  arise,  how  do  we  define  “Operational  Analysis”  and  who 
should  be  considered  as  an  operational  analyst? 

This  dispersion  of  OA  will  not  guarantee  the  professional  standing  and  development  of  the  OA  practices. 
Additionally,  including  generic  military  analysts  without  a  professional  education  or  background  in  OA  will 
further  affect  the  quality  of  headquarters’  OA  capability,  to  the  extent  that  questions  arise  whether  or  not  OA 
is  present  in  the  HQ. 

5.5.1  Professional  Communities 

Military  planners,  military  logisticians,  doctors,  teachers,  lawyers  and  engineers  have  professional 
communities.  There  is  a  large  and  established  body  of  people  whose  profession  is  the  application  of  scientific 
and  structured  analysis  to  support  decision-making  in  various  fields  in  defence  and  the  need  has  been 
expressed  by  many  to  have  an  established  professional  community  that  can  allow  the  following: 

•  Establish  a  professional  identity  with  an  external  component  -  recognition  by  customers  -  and  an 
internal  component  -  self-identification; 

•  Education,  training  and  professional  development; 

•  Networking; 

•  Collaboration  on  projects; 

•  Technical  development,  to  include  Identification  and  definition  of  shared  theories  and  concepts;  Best 
Practices  and  setting  technical,  ethical  standards  and  values;  and  mechanism  to  share  (and  validate) 
work  of  common  interest  to  members  of  the  community; 

•  Identification  of  issues  of  future  importance; 

•  Community  support  to  include  representation,  moral  &  social  support,  and  rewarding  and  recognising 
achievement. 

5.5.2  An  Analysis  Community? 

We  propose  that  the  field  of  ‘Operational  Analysis’  be  considered  as  those  professionally  trained  analysis  staff 
that  are  experts  in  applying  rigorous,  structured,  scientific  methods  to  understand  complex  problems  to  assist 
decision-makers  within  the  military  domain  with  independent  advice. 
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There  are  significant  changes  in  the  OA  staffs  in  NATO:  in  ACO,  OA  branches  are  being  disbanded  and 
dispersed  amongst  Assessment,  Knowledge  Development,  Plans  and  Command  Group  branches;  in  NC3A, 
there  is  a  project  focus  in  which  the  OR  Division  has  been  replaced  by  a  loosely  defined  ‘OR  Community’; 
and  in  ACT  the  OA  branch  has  been  reduced  to  50%  manning  in  the  new  PE  structure. 

There  is  sufficient  evidence  to  suggest  that  OA  posts  in  Knowledge  Development,  Plans  and  Assessment  are 
still  necessary  and  that  additional  analysis  functions  can  be  justified,  such  as  political,  social,  and  economic 
analyses.  The  question  is  here,  whether  these  other  analysis  functions  should  be  included  into  the  OA 
community  to  form  a  wider  “Analysis  Community”. 

OA  functionality  within  in  NATO  Commands  is  needed,  whether  this  is  for  support  to  operations,  derivation 
of  lessons,  transformational  activities  in  CD&E,  requirement  derivation  or  events  analysis.  The  need  to  form 
an  OA  community  is  enhanced  as  OA  becomes  more  dispersed  within  the  new  PE  structure. 


6  2010  OA  WORKSHOP 

On  15-17  June  2010,  HQ  SACT  will  be  hosting  the  2010  Operational  Analysis  (OA)  Workshop.  This  three 
day  workshop  will  focus  on  methods  of  analysis  to  develop  concepts,  and  aims  to  bring  together  Analysts  and 
Concept  Development  practitioners  in  order  to  examine  the  problems  that  Concept  Developers  face  and  the 
role  of  analysis  in  supporting  Concept  Development.  Within  NATO,  Concepts  are  a  primary  mechanism  for 
initiating  and  debating  change,  yet  in  comparison  to  Experiments,  it  has  been  challenging  to  apply  a  wide 
variety  of  analytical  techniques.  It  is  the  workshop’s  intention  to  determine  analysis  requirements  and  fit 
analytical  methods  to  problems.  The  outputs  will  be  used  to  inform  best  practice  for  Concepts  Development 
within  NATO.  Again  nations  have  been  invited  to  attend  the  workshop  and  to  present  their  ideas  on  Concept 
Development.  The  workshop  will  be  a  mix  of  plenary  and  breakout  sessions.  Issues  to  be  discussed  will 
include: 


•  Case  studies  from  NATO  and  national  Concept  Development  centres; 

•  The  types  of  issues  and  problems  facing  Concept  Developers; 

•  The  state-of-the-art  with  respect  to  tools  and  techniques  for  analysing  a  Concept; 

•  Opportunities  for  Operational  Analysis  in  supporting  Concept  Development; 

•  The  challenges  facing  analysts  conducting  Concepts  analysis  at  various  conceptual  levels  -  Capstone, 
Operating,  and  Functional. 


7  CONCLUSIONS 

The  OA  Workshops  fulfil  a  need  in  the  NATO  analysis  community  to  gather  on  an  annual  basis  the 
practitioners  of  analysis  within  the  NATO  commands,  centres  and  agencies,  augmented  by  many  OA  analysts 
from  the  national  laboratories,  centres  and  organizations.  The  OA  workshop  started  out  to  allow  for  a  general 
display  of  what  the  participants  were  engaged  in  and  worked  on  in  their  parent  organizations,  but  quickly 
devoted  itself  to  discuss  specific  topics,  reaching  out  to  the  military  staffs  to  allow  their  participation  to  grow 
understanding  and  awareness  of  analytical  issues. 

The  promises  of  the  first  OA  workshop  in  2007  to  develop  an  OA  handbook  were  shelved  when  it  became 
clear  that  the  first  priority  of  the  NATO  OA  community  was  to  survive  the  restructuring  of  the  NATO  Peace 
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Establishment  Review,  and  that  attempts  to  build  processes  and  procedures  on  OA  could  only  disenchant  the 
leadership  as  they  were  asking  questions  on  the  value  that  OA  brings  to  their  staff  work.  It  must  be  assessed 
whether  subsequent  re-arrangement  and  re-assignment  of  analysts  to  branches  not  solely  oriented  on  OA  may 
have  done  the  OA  cause  good  or  has  been  detrimental  to  it.  Attempts  to  build  a  consolidated  Analysis 
Requirements  List  were  diverted  when  it  became  clear  that  the  list  was  only  meant  to  build  a  program  of  work 
for  the  JALLC.  Several  steps  have  been  set  to  build  a  NATO  OA  community  and  to  conduct  analysis  training. 
Firstly,  the  OA  workshop  is  now  in  an  annual  rhythm  and  it  is  the  intention  of  the  OA  in  HQ  SACT  to 
continue  to  provide  this  forum.  Additionally,  a  full-week  analysis  course  exists  for  JALLC  staffers 
introducing  them  to  analysis  tasks  they  need  to  perform  when  tasked  to  investigate  operational  issues  within 
the  NATO  operational  theatres  or  during  exercises.  The  NATO  CD&E  course  at  NATO  School 
Oberammergau  contains  three  modules  on  analysis:  introduction,  concepts  analysis  and  experimentation 
analysis.  The  NC3A  has  build  an  Operations  Research  Peer  Network,  that  functions  as  an  internal  entity  for 
discussion,  training,  education  and  invites  guest  speakers  to  talk  about  special  topics  in  Operations  Research. 
Plans  to  extend  the  NATO  School  curriculum  with  an  introductory  course  on  appreciation  of  Operational 
Analysis  have  been  voiced  but  are  still  in  the  embryonic  phase. 

The  2008  and  2009  OA  Workshops  have  been  well-attended  and  highly  appreciated,  particularly  by  the 
military  staffers  that  either  sought  a  forum  to  discuss  a  topic  on  its  analytical  merits,  or  had  a  desire  to  broaden 
their  knowledge  on  analysis  and  gain  exposure  to  the  wider  NATO  OA  community.  Fall-out  and  follow-on 
actions  from  the  2008  workshop  on  assessment  allowed  for  clearer  formulation  of  what  assessment  in 
operations  planning  needed  to  be.  The  2009  OA  workshop  raised  the  awareness  of  some  decision  makers  on 
what  OA  stood  for.  The  session  on  systems  analysis  and  operations  planning  during  this  workshop  was  very 
well  attended  and  allowed  for  discussion  between  analysts  and  military  staff  on  the  current  issues,  but  also 
gave  the  opportunity  to  display  national  viewpoints  and  opinions.  The  discussion  on  experimentation  allowed 
for  the  management  of  expectations  from  experimentation  projects  in  ACT,  and  has  given  rise  to  possibilities 
for  experimentation  looking  less  like  exercises,  and  more  like  seminars  and  war  games.  The  prime  example  of 
that  is  the  Concept  Development  Assessment  Game  (CD AG)  on  Maritime  Situational  Awareness  that  will  be 
held  in  May  with  assistance  from  the  SAS-086  Specialist  Team. 

A  lesson  that  was  learned  from  the  JCOA  specialists  was  that  engagement  of  the  stakeholder  in  the  analysis 
task  is  vital  for  the  acceptance  of  the  recommendations  that  fall  out  of  the  analysis,  whether  it  is  in 
experimentation,  in  exercises  or  in  operations.  Efforts  to  engage  our  leadership  to  understand  what 
Operational  Analysis  brings,  is  vital  for  our  profession.  Equally  important  is  the  engagement  of  the  analyst  in 
the  project,  topic  or  issue  for  which  analysis  is  provided.  Close  proximity  to  the  military  staff  that  needs  to  be 
supported  is  needed,  so  the  question  arises  whether  the  restructuring  of  the  OA  in  ACO  and  re-assignment  into 
other  branches  will  turn  out  well  for  the  cause  of  analysis. 

A  continuing  engagement  with  and  support  to  the  military  staff  by  the  OA  analyst  has  been  identified  as  the 
prime  reason  for  a  steady  request  for  Operational  Analysis  in  any  form.  While  initially  this  support  may  not 
immediately  be  focused  on  solving  problems  with  analytical  methods  and  tools,  the  inclusion  of  the  OA 
analyst  in  projects  will  guarantee  the  infusion  of  logic  and  subsequently  analytical  methods  and  scientific 
reasoning  will  make  their  way  into  the  project’s  work  allowing  it  to  be  based  on  sound  principles  and  common 
rules.  It  may  be  that  in  operations  and  exercises,  the  analytical  support  will  have  to  be  immediate  and  to  the 
point  to  be  effective,  however,  in  longer  term  projects  such  as  experiments  or  concept  development,  there  is 
ample  time  to  inject  analysis. 
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